arXiv:1508.06257vl [cs.IR] 25 Aug 2015 


Prediction of Cyberbullying Incidents on the Instagram 

Social Network 

Homa Hosseinmardit, Sabrina Arredondo Mattson^, Rabat Ibn Rafiq*, 

Richard Han*, Qin Lv*, Shivakant Mishra* 

^Computer Science Department, institute of Behavioral Science 
University of Colorado Boulder, Boulder, CO 80309 USA 
homa.hosseinmardi@colorado.edu 


ABSTRACT 

Cyberbullying is a growing problem affecting more than half 
of all American teens. The main goal of this paper is to in¬ 
vestigate fundamentally new approaches to understand and 
automatically detect and predict incidents of cyberbullying 
in Instagram, a media-based mobile social network. In this 
work, we have collected a sample data set consisting of In¬ 
stagram images and their associated comments. We then de¬ 
signed a labeling study and employed human contributors at 
the crowd-sourced CrowdFlower website to label these me¬ 
dia sessions for cyberbullying. A detailed analysis of the 
labeled data is then presented, including a study of relation¬ 
ships between cyberbullying and a host of features such as 
cyberaggression, profanity, social graph features, temporal 
commenting behavior, linguistic content, and image content. 
Using the labeled data, we further design and evaluate the 
performance of classifiers to automatically detect and pre¬ 
dict incidents of cyberbullying and cyberaggression. 
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J.4 [Social And Behavioral Sciences]: Sociology; H.3.5 
[Online Information Services]: Web-based Services 
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Design, Human Factors, Measurement 
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1. INTRODUCTION 

As online social networks (OSNs) have grown in popu¬ 
larity, instances of cyberbullying in OSNs have become an 
increasing concern. In fact more than hal f of American teens 
have been the victims of cyberbullying 35 . Although cy¬ 


berbullying may not cause any physical damage initially, it 
has potentially devastating psychological effects like depres¬ 
sion, low self-esteem, suicide ideation, and even suicide 
16 13 ^ and can have long term effects in the future life 
OT victims [^. Incidents of cyberbullying with extreme con¬ 
sequences such as suicide are now routinely reported in the 
popular press. For example cyberbullying of Jessica Logan 
via her image shared in Facebook and MySpace and of Hope 
Sitwell with her image shared in MySpace is attributed to 


their suicides [^, [36| . Also, nine teenagers reportedly com¬ 
mitted suicide having experienced cyberbullying in Ask.fm 
1^. Although cyberbullying is not the direct cause of these 
suicides, it was viewed as a contributing factor in the death 
of these teenagers . 

Given the gravity of the consequences cyberbullying has 
on its victims and its rapid spread among middle and high 
school students, there is an immediate and pressing need for 
research to understand how cyberbullying occurs in OSNs 
today, so that effective techniques can be developed to ac¬ 
curately detect cyberbullying. In [^, it is reported that 
experts in the field of cyberbullying could favor automatic 
monitoring of cyberbullying on social networking sites and 
propose effective follow-up strategies. 

Our work makes the important distinction between cy¬ 
beraggression, which concerns the aggressiveness of a single 
remark toward a user, and eyberbullying, which concerns the 
overall pattern of aggressiveness of many remarks directed 
at a user. It is this pattern of aggression that severely im¬ 
pacts many teens. In fact, cyberbullying has been defined 
as intentionally aggressive behavior that is repeatedly earried 
out in an online context against a person who cannot easily 
defend him or herself [26[|40| . It is important to this defini¬ 
tion of cyberbullying that both the frequency of negativity 
and the imbalance of power between the victim and perpe¬ 
trator be taken into account. In contrast, cyberaggression is 
defined as an instance of using digital media to intentionally 
harm another person [26| . 

Facebook, Twitter, YouTube, Ask.fm, and Instagram have 
been listed as the top five networks with the highest per¬ 
centage of users reporting experience of cyberbullying [11| . 
Instagram is of particular interest as it is a media-based 
mobile social network, which allows users to post and com¬ 
ment on images. Cyberbullying in Instagram can happen 
in different ways, including posting a humiliating image of 
someone else by perhaps editing the image, posting mean or 
hateful comments, aggressive captions or hashta gs, or creat¬ 
ing fake profiles pretending to be someone else [^. Figure 
[^illustrates an example of an attack in Instagram in which 
hateful comments were posted for the profile owner. 

The main goal of this paper is to first study cyberbullying 
in Instagram and then develop classifiers to automatically 
detect cyberbullying incidents and predict onset of cyberbul¬ 
lying incidents. To do so, we first collected a large sample 
of Instagram data comprised of 3,165K media sessions (im¬ 
ages and their associated comments) taken from 25K user 
profiles. Next, we designed and conducted a set of labeling 
surveys using human labelers at the crowd-sourced Crowd- 
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You dll are really wastini 
he dont care. Get over 
pos^hateveMi^want^ 


r 


Disrespectful. 


Hey. do the world a favor and go kill yourself. 


I M^Manne^woul^ho^oiynt^iece^ 


I ahahah this is funny. You don't have what it 

takes to wear my uniform. Do me a favor, next time 
you wanna wear that come over to camp pen and 
wear it in front of me and my other grunts. But you 
dorVUjav^h^ball^^cHik^^nan^o you play 
dress up and pretend to be. #LikeA6itch 


Figure 1: An example of comments posted on In- 
stagram. To give more room for the text, we have 
moved the associated image to overlay some of the 
text. 


Flower website to identify occurrences of cyberbullying and 
cyberaggression in Instagram. We then analyzed the labeled 
data set, reporting different features of these media sessions 
with respect to cyberbullying. Finally, based on our analy¬ 
sis, we have designed and evaluated classifiers for detecting 
cyberbullying incidents and predicting onset of cyberbul¬ 
lying incidents. This paper make the following important 
contributions: 

• We provide a clear distinction between cyberbullying 
and general cyberaggression. Cyberbullying is a stricter 
form of cyberaggression, while most of the earlier re¬ 
search in this area has focused on identifying cyberag¬ 
gression. 

• We obtain ground truth about cyberbullying behavior 
in Instagram by asking human crowd-sourcers to la¬ 
bel Instagram images and their associated comments 
according to both the more restrictive definition of cy¬ 
berbullying and the more general definition of cyber¬ 
aggression. Labelers are provided with the image and 
its associated comments at the same time to be able 
to understand the context and label accordingly. 

• We present a novel detailed analysis of the labeled me¬ 
dia sessions, including the relationships between cy¬ 
berbullying and a host of factors, such as cyberaggres¬ 
sion, profanity, social graph properties (liking, follow¬ 
ers/following), the interarrival time of comments, the 
linguistic content of comments, and labeled image con¬ 
tent. 


• Finally, we design and evaluate multi-modal classifiers 
to predict cyberbullying based on the labeled data. 
To the best of our knowledge, no other research has 
addressed the problem of predicting cyberbullying in 
social networks. 
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2. RELATED WORKS 

Prior works that investigated cyberbullying [43| |22| 

51 ^ 29 10 30 are more accurately described as 
research on cyberaggression, since these works do not take 
into account both the frequency of aggression and the im¬ 
balance of power. These works have largely applied a text 
analysis approach to online comments, since this approach 
results in higher precision and lower false p osit ives than sim¬ 
pler list-based matching of profane words [47] . Previous re¬ 
search 33 34 24 applied text based cyberbullying on 
Formspring.me and Myspace dataset. Dinakar el al. investi¬ 
gated both explicit and implicit cyberbullying by analyzing 
negative text comments on YouTube and Formspring pro¬ 
files [^. Sanchez and Kumar proposed using a Naive Bayes 
classifier to find inapp rop riate words in Twitter text data 
for bullying detection 
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They tracked potential bullies, 
their followers, and the victims. Also some researchers tried 
to detect bullies and victims by looking at the number of re¬ 


ceived and sent, beside detecting aggressive comments 32 


and 30 . Dadvar et al. investigated how combining text 


analysis with MySpace user profile information such as gen¬ 
der can improve the accuracy of cyberbullying detection in 
OSNs . Huang et al. has consider some graph proper¬ 
ties besides text features, however they also worked only over 
comment-based labeled data. Another work has looked at 
the time series of posted comments of Formespring dataset, 
which each question answer pair was label ed s eparately as 
cyberaggression and predict their severity, 42 . Also much 
of these works have considered text-based approach to an¬ 
alyze comments in OSNs instead of analyzing the features 
associated with the media objects such as images or videos 
belonging to those comments. Kansara et al. suggest 
only a framework for using images beside text for detecting 
cyberbullying. 

Other work analyzed profanity usages in Instagram 
and Ask.fm 17 comments, but did not label the data at all. 
Our previous work, 18 detects cyberbullyning incidents in 
the Instagram for images with comments with more than 
40% negativity. Additional research investigated aspects of 
the Instagram social network, but not in the context of cy¬ 
berbullying. For example, explored users’ photo shar¬ 
ing experience in a museum. Silva et al. considered the 
temporal photo sharing behavior of Instagram users and Hu 
et al. |52| categorized Instagram images into eight popular 
image categories and the Instagram users into five types in 
terms of their posted images. By investigating user prac¬ 
tices in Instagram, concluded that users tend to be more 
active during weekends and at the end of the day. They also 
found out that users are more likely to like and comment on 
the medias that are already popular, thereby inducing the 
rich get richer phenomenon. 


• We design and evaluate multi-modal classifiers that 
extend beyond merely the text dimension for detect¬ 
ing cyberbullying and further incorporate image-based 
features and user statistics in the detection. 


3. DATA COLLECTION 

Starting from a random seed node, we identified 41K In¬ 
stagram user ids using a snowball sampling method from 
the Instagram API. Among these Instagram ids, 25K (61%) 
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users had public profiles while the rest had private profiles. 
Due to the limitation on the private profiles’ lack of shared 
information, the 25K public user profiles comprise our sam¬ 
ple data set. For each public Instagram user, the collected 
profile data includes the media objects (videos/images) that 
the user has posted and their associated comments, user id 
of each user followed by this user, user id of each user who 
follows this user, and user id of each user who commented on 
or liked the media objects shared by the user. We consider 
each media object plus its associated comments as a media 
session. 

Labeling data is a costly process and therefore in order 
to make the labeling of cyberbullying more manageable, we 
sought to label a smaller subset of these media sessions. To 
have a higher rate of cyberbullying instances, we considered 
media sessions with at least one profanity word in their asso¬ 
ciated comments. W e ta g a comment as “negative” using an 
approach similar to [^. For this set of 25K users, 3,165K 
unique media sessions were collected, where 697K of these 
sessions have at least one profane word in their comments 
by users other than the profile owner, where a profane word 
is obtained from a dictionary 37 , |49|. 


In addition, we needed media sessions with enough com¬ 
ments so that labelers could adequately assess the frequency 
or repetition of aggression, which is an important part of the 
cyberbullying definition. We selected a threshold of 15 as 
a lower bound on the number of comments in a media ses¬ 
sion, considering that the average ratio of comments posted 
by users other than friends to comments posted_^ the pro¬ 
file owner in an Instagram profile is around 16 19 . At the 


end 2,218 media sessions (images and their associated com¬ 
ments) were selected randomly for the task of labeling. 


investigate cyberaggression and cyberbullying in this multi¬ 
modal context (textual comments and media objects). Our 
labeling process consisted of two separate surveys. The first 
survey incorporates both the image and the associated text 
comments when asking the human contributors whether the 
media session was an instance of cyberbullying or cyberag¬ 
gression. The second survey is focused on the image contents 
alone and is used to identify the content and category of the 
image. 

Figure [^illustrates an example of our design for the first 
labeling survey. On the left is the image, and on the right 
is a scrollable interface to help the contributor see all of the 
comments associated with this image. With input from a so¬ 
cial science expert, we designed a simple survey that asked 
the contributors two questions, namely whether the media 
session constituted cyberaggression or not, and whether the 
media session constituted cyberbullying or not. During the 
instructional phase prior to labeling, contributors were given 
the aforementioned definitions of cyberaggression and cyber¬ 
bullying along with related examples. Each media session 
was labeled by five contributors. 



Media posted at 012-02-13 12:38:58 


that muhh (created at:2012-02-1312:59:19) 


4. CYBERBULLYING LABELING 

In this section, we explain the design and methodology 
for our survey for labeling the selected set of media ses¬ 
sions. Our first challenge is choosing appropriate definitions 
of terms, which will then be used in ground truth labeling. 
Based on the literature, a major early choice that we have 
made is to distinguish between cyberaggression and cyber¬ 
bullying. Cyberaggression is broadly defined as any occur¬ 
rence o f us ing digital media to intentionally harm another 
person 26 . Examples include negative content and words 
such as profanity, slang and abbreviations that would be 
used in negative posts such as hate, fight, wtf. Cyberbully¬ 
ing is one form of cyberaggression that is more restrictively 
defined as intentional aggression that is repeatedly carried 
out in an electronic context against a person who cannot 
easily defend him or herself 26 Thus, cyberbullying 

consists of three main features : (1) an act of aggression 
online; (2) an imbalance of power between the individuals 
involved; and (3) it is repeated over time [21 
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The power imbalance can take on a variety of forms inc 
ing physical, social, relational or psychological |12| 28 
|44| , such as a user being more technologically savvy than 
another 25 , a group of users targeting one user, or a pop¬ 
ular user targeting a less popular one |^. Repetition of 
cyberbullying can occur over time or by forwarding/sharing 
a negative comment or photo with multiple individuals [^. 

In Instagram, each media session consists of a media ob¬ 
ject posted by the profile owner and the corresponding com¬ 
ments for the media object. The goal in this paper is to 


Is there any cyberaggressive behavior in the online posts? Mark yes if there is at least one 
negative word/comment and or content with intent to harm someone or others. 

_i No 
OYes 

Is there any cyerbullying in the online post? Mark yes if there are negative words and or 
comment with intent to harm someone or other, and the posts include two or more 
repeated negativity against a victim that cannot easily defend him or herself 
O No 
OYes 


Figure 2: An example of the labeling survey, show¬ 
ing an image and its corresponding comments, and 
the survey questions. 


4.1 Quality Control 

In order to provide quality control, we only permitted the 
highest-rated contributors on CrowdElower to have access 
to our job. Next, a mentoring phase was provided for the 
potential contributors that included instructions and a set 
of example media sessions with the correct label. Further, 
to monitor the quality of the contributors and filter out the 
spammers, potential contributors were asked to answer a 
set of test questions in two phases: quiz mode and work 
mode. Potential contributors needed to answer correctly a 
minimum number of test questions to pass the quiz mode 
and qualify as a contributor for the survey. We also incor¬ 
porated quality control checks during the labeling process 
(work mode) by inserting random test questions. A contrib¬ 
utor was filtered out if he/she failed this work mode. Finally, 
a minimum time threshold was set to filter out contributors 
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who rushed too quickly through the labeling process. The 
minimum number of test questions to get back high-quality 
data was recommended by CrowdFlower. 

Overall, 176 potential contributors worked on the quiz 
questions, 144 passed the quiz mode, while 31 contributors 
failed and 1 gave up. The labeled data that we finally ob¬ 
tained were from 139 trusted contributors, while the the rest 
were filtered out during the work mode. Table[^provides the 
number of trusted judgments and the contributors’ accuracy 
for 11,090 total judgments. 

Table 1: Labeling process statistics. Trusted judg¬ 
ments are the ones made by trnsted contribntors. 


Trusted Judgments 

10987 

Untrusted Judgments 

103 

Average Test Question Accuracy of 

89% 

Trusted Contributors 


Labeled Media Sessions per Hour 

6 


4.2 Image Labeling 

Next, we conducted a second separate survey just to label 
the image contents of media sessions, so that we could in¬ 
vestigate the relationship between cyberbullying and image 
content. We first sampled 1,200 images from the selected 
subset of media sessions to determine a suitable set of rep¬ 
resentative categories to be used in the labeling. A grad¬ 
uate student examined all the images and classified them 
to different possible categories. Then for designing the sur¬ 
vey, a social science expert checked the categories again and 
revised them. Some of the dominant categories identified 
were the presence of a human in the image, as well as text, 
clothes, tattoos, sports and celebrities. We then asked con¬ 
tributors to identify which of the aforementioned categories 
were present in the image. Multiple categories could be se¬ 
lected for a given image. Each media session was labeled by 
three different contributors. 

At the end, our social science expert checked a set of 
random media sessions and images to confirm the quality of 
the labeled data for both surveys. 

5. ANALYSIS AND CHARACTERIZATION 
OF GROUND TRUTH DATA 

We submitted our first survey with 2,218 media sessions 
(images and their associated comments) to CrowdFlower, 
a crowd-sourced website. Each media session was labeled 
by five different contributors. CrowdFlower provides a de¬ 
gree of trust for each contributor based on the percent¬ 
age of correctly answered test questions explain in section 
1^ This trust value is incorporated by CrowdFlower into 
a weighted version of the majority voting method called a 
“confidence level” for each labeled media session. We de¬ 
cided to keep media sessions whose weighted trust-based 
metric was equal to or greater than 60%. We deemed them 
to be strong enough support for majority voting from con¬ 
tributor with higher trust. Overall, 1,954 (88%) of the orig¬ 
inal pure majority-vote based media sessions wound up in 
this higher-confidence cyberbullying-labeled group. For this 
higher-confidence data set, 29% of the media sessions be¬ 
longed to the “bullying” group while the other 71% were 


deemed to be not bullying. 

5.1 Labeling and Negativity Analysis 

Distribution of the media sessions based on the num¬ 
ber of votes (out of five votes) received for cyberaggression 
and cyberbullying respectively has been provided in Fig¬ 
ure The left chart shows the fraction of samples that 
have been labeled as cyberaggression k times, and the right 
chart shows the fraction of samples that have been labeled 
as cyberbullying k times. Higher the number of votes for 
a given media session, more confidence we have that the 
media session contains an incident of cyberaggression or cy¬ 
berbullying, with five votes means unanimous agreement. 
Similarly, lower the number of votes for a given media ses¬ 
sion, more confidence we have that the media session does 
not contain an incident of cyberaggression or cyberbullying, 
with zero votes means unanimous agreement.The inter-rater 
agreement Fleiss-Kappa value for cyberbullying is 0.5 and 
for cyberaggression is 0.52. 



k votes k votes 

Figure 3: Fraction of media sessions that have been 
voted k times as cyberaggression (left) or cyberbul¬ 
lying (right). 

We notice that for both cyberaggression and cyberbully¬ 
ing, most of the probability mass is around media sessions 
labeled by all four or five contributors the same, i.e. either 
0 or 1 votes (about 50% for cyberaggression and about 62% 
for cyberbullying), or 4 or 5 votes (about 31% for cyber¬ 
aggression and about 23% for cyberbullying). Thus, a key 
finding is that the contributors are mostly in agreement about 
what behavior constitutes cyberaggression, and what behavior 
constitutes cyberbullying in Instagram media sessions. Only 
about 13—17% of the media sessions have two or three votes, 
which indicates that there is some disagreement in a small 
fraction of media sessions about whether the session con¬ 
tains an incident of cyberaggression or cyberbullying. This 
disagreement can be attributed to the fact that different 
people have different levels of sensitivity and a conversation 
may seem normal to one person and hurtful to another. 

Next, we observe that about 30% of the media sessions 
have not been labeled as cyberaggression by any of the five 
contributors. Since all media sessions contained at least one 
comment with one or more profane word, this suggests that 
only employing a profanity usage threshold to detect cyber¬ 
aggression can produce many false positives. We make a 
similar observation for cyberbullying. We notice about 40% 
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of the media sessions have not been labeled as cyberbullying 
by any of the five contributors. Applying a majority voting 
criterion to a binary label as cyberbullying or not, 30% of 
the samples have been labeled as cyberbullying. This is de¬ 
spite the fact that all the media sessions contain at least 
one profane word. This leads us to our second important 
finding. A classifier design for cyberbullying detection can¬ 
not solely rely on the usage of profanity among the words 
in image-based discussions, and instead must consider other 
features to improve accuracy. 


0.25 

0.2 

0.15 

0.1 

0.05 

0 

0 1 2 3 4 5 

k cyberbullying votes 

Figure 4: Two-dimensional distribution of media 
sessions as a function of the number of votes given 
for cyberaggression versus the number of votes given 
for cyberbullying, assuming five labelers. 

In order to understand the relationship between cyberag¬ 
gression and cyberbullying, we plotted in Figure a two- 
dimensional heat map that shows the distribution of me¬ 
dia sessions as a function of the number of votes each me¬ 
dia session received for cyberaggression and cyberbullying. 
We observe that a significant fraction of the sessions exhibit 
strong agreement in terms of either receiving high numbers 
of votes for both cyberbullying and cyberaggressions, or re¬ 
ceiving low numbers of votes for both cyberbullying and 
cyberaggression. This can be inferred from the high energy 
in the upper right and lower left part of the diagonal. In 
addition, it is promising that the area below the diagonal is 
essentially zero, meaning no session has received more votes 
for cyberbullying than for cyberaggression. This conforms 
with the dehnition that cyberbullying is a subset of cyberag¬ 
gression. The Pearson’s correlation between number of vote 
for cyberbullying and number of votes for cyberaggression 
is 0.9. 

We see that the remaining significant energy in the dis¬ 
tribution appears in the area above the diagonal. Media 
sessions in this area exhibit the property that if they receive 
Ni cyberbullying votes and N 2 cyberaggression votes, then 
N 2 > Ni. The area where Ni < 2 and N 2 > 3 corresponds 
to cases where there is cyberaggression but not cyberbul¬ 
lying. These observations lead us to our third important 
finding. A media session that exhibits cyberaggression does 
not necessarily exhibit cyberbullying, and a classifier design 
for cyberbullying detection must go much beyond merely de¬ 
tecting cyberaggression. This is a very important finding, 
because as we noted in Section prior work on detecting 
cyberbullying has mainly focused on detecting cyberaggres¬ 



sion as they do not take into account the frequency of ag¬ 
gression or imbalance of power, which are crucial features of 
cyberbullying. 

Finally, we are interested in understanding the relation 
between cyberbullying/cyberaggression and the percentage 
of negativity in the comments. We divided all the media 
sessions into nine different bins based on the percentage of 
negativity in their comments. Bin (rii — 112 ] contains all me¬ 
dia sessions with bigger than ni% and smaller than or equal 
to n 2 % negativity. None of the media sessions contained 
more than 90% negative comments. Next, we calculated per¬ 
centage media sessions for each bin that can be identified as 
cyberaggression or cyberbullying based on majority of votes, 
i.e. where the number of votes is 3 or higher. Figure|^shows 
these fractions, left figure for cyberaggression and right fig¬ 
ure for cyberbullying. We observe that as the percentage of 
negativity increases, so does the fraction of media sessions 
up until 50% negativity for cyberaggression and 60% for cy¬ 
berbullying. This increase is as expected, since cyberaggres¬ 
sion or cyberbullying is typically accompanied with negativ¬ 
ity in the postings. However, we notice that the percentage 
of cyberaggression or cyberbullying starts decreasing after 
these peaks as the percentage of negativity increases. This 
is quite an unexpected result and seems counter-intuitive. 
To understand this, we examined closely the media sessions 
that have very high negativity. We noticed that these media 
sessions typically involved discussions about sports, politics, 
tattoos, or were just friendly talks. People tend to use lots 
of profanity words in such discussion, even though they are 
not insulting any one person in particular. This leads us 
to our final important hnding about negativity analysis. A 
media sessions with a significantly high percentage of nega¬ 
tivity (more than 60-70%) typically implies a low probability 
that the session contains a cyberbullying incident. 


Cvberaaaression Cvberbullvina 



Percentage of negativity Percentage of negativity 


Figure 5: Percentage of media sessions that have 
been labeled as cyberaggression (left) and cyberbul¬ 
lying (right) versus their negativity percentages. 


5.2 Temporal and Graph Properties Analysis 

Since different comments in a media session are posted 
at different times, it is important to understand the rela¬ 
tionship between the temporal nature of comment postings 
and cyberbullying/cyberaggression. We define the strength 
of cyberbullying/cyberaggression as the number of votes re¬ 
ceived for labeling a media session as cyberbullying/cyber- 
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aggression, and explore the Pearson’s correlation of cyber- 
bullying/cyberaggression strength and temporal behavior 
comment arrivals. We would like to understand how hu¬ 
man contributors incorporated the dehnition of cyberbul- 
lying, which includes the temporal notion of repetition of 
negativity over time, into their labeling. Given the time 
stamps on the collected comment, we compute the inter¬ 
arrival time between two consequent comments. We then 
count the number of interarrival times of comments in a 
media session that have a value less than x = Imin, 5 min, 
...,6 months. 

Figure illustrates the correlation between the number 
votes and the number of comments arrive with less than 
or equal x seconds after their previously received comment. 
We see that there is a correlation of about 0.3 between the 
strength of support for cyberbullying and media sessions in 
which there are frequent postings within one hour of previ¬ 
ous post. Further, we find that as we expand the allowable 
interarrival times between comments, the correlation weak¬ 
ens considerably. Similar pattern was observed for cyberag¬ 
gression. In fact, on average around 40% of the comments 
arrive in less than 1 hours after previously received com¬ 
ment in cyberbullying media sessions, however only 30% of 
the comments have been received with the same interarrival 
time in non-cyberbullying samples (p < 0.001, based on t- 
test). A key finding here is that media sessions that contain 
cyberbullying have relatively low comment interarrival times, 
that is the comments in these media sessions are posted quite 
frequently. 



Figure 6: Pearson’s correlation between the number 
of votes and the number of comments arrive with 
less than or eqnal x seconds after their previously 
received comment. (X-axis has been converted to 
coarser time units than second) 

Next, we examine the relationship between cyberbully- 
ing/cyberaggression and the social network graph features 
such as the number of likes for a given media object, num¬ 
ber of comments posted for a media object, number of users 
a user is following, and the number of followers of a user. Ta- 
ble[^shows these numbers, for categories of non-cyberbullying 
sessions, cyberbullying sessions, non-cyberaggression sessions 
and cyberaggression sessions. We observe that media ses¬ 
sions that contain cyberbullying/cyberaggression share more 
media objects than media sessions that do not contain cy¬ 
berbullying/cyberaggression, but on average receive lower 
number of likes. Souza et al.’s analysis of Instagram 
users shows there is a positive correlation between num¬ 
ber of followers and number of likes for typical Instagram 
users. Users who receive cyberbullying do not follow the 


same pattern. In fact, the average number of likes per post 
for non-cyberbullying sessions is 4 times the average num¬ 
ber of likes for cyberbullying sessions, and the the average 
number of likes per post for non-cyberaggression sessions 
is 4.5 times the average number of likes for cyberaggres¬ 
sion sessions. In terms of number of following and followers, 
the distinction is not as pronounced, although we see that 
the media sessions with cyberbullying/cyberaggression in¬ 
cidents have more followers and less following compared to 
the media sessions without cyberaggression/cyberbullying. 
The key finding here is that the users of media sessions with 
cyberbullying/cyberaggression have lower number of likes per 
post while have more followers. 


5.3 Linguistic and Psychological Analysis 

We now focus on the pattern of linguistic and psycholog¬ 
ical measurements of cyberbullying/cyberaggression media 
sessions versus non-cyberbullying/non-cyberaggression. For 
this purpose, we have applied Linguistic Inquiry and Word 
Count (LIWC) a text analysis program to find which cate¬ 
gories of words have been used for cyberbullying/cyberaggre¬ 
ssion labeled media sessions. LIWC evaluates different as¬ 
pects of word usages in psychologically meaningful cate¬ 
gories, by countin g th e number of the words across the text 
for each category 41 . LIWC has often been used for studies 


on variations in language use across different people. Pub¬ 
lished papers show that LIWC have been validated to per¬ 
form well in studies on variations in language use across 
different peoples [^. We first analyze the number of words, 
and usage of pronouns, negations and swear words (Fig¬ 
ure]^. Next, we look at some of the personal concerns such 
as work, achievements, leisure, etc. (Figure [^. Finally, 
we investigate some of the psychological measurements such 
as social, family, friends, etc. (Figure]^. For each of these 
cases, we first obtain the LIWC values for each media session 
comment set. We then calculate the average LIWC value 
for each of the four classes: media sessions with cyberbul¬ 
lying, media sessions with no cyberbullying, media sessions 
with cyberaggression, and media sessions with no cyberag¬ 
gression. The bars shown in Figures [7|9| represent the ratio 
of average LIWC value for cyberbullying class to that of 
non-cyberbullying, and the ratio of average LIWC value for 
cyberaggression class to that of non-cyberaggression. 

In Figure we first notice that the word count for me¬ 
dia sessions with cyberbullying/cyberaggression is signifi¬ 
cantly higher than for media sessions with no cyberbully¬ 
ing/cyberaggression (p < 10“®). Next, as expected, for 
swear words (e.g., damn, piss) and negations (e.g., never, 
not), the ratio is higher for cyberbullying/cyberaggression 
category (p < 10“®). It is interesting to note that the ratios 
for the third person pronouns (she, he, they) are more than 
1.3 (p < 10“®), ratio for the first person singular pronoun ( 
i ) is 0.85, and the ratios for hrst person plural and second 
person pronouns (we, you) is closer to 1. This leads us to 
our first key finding with respect to the linguistic features. 
A user is less likely to directly refer to himself/herself and 
more likely to refer to other people in third person in postings 
involving cyberbullying or cyberaggression. 

For personal concerns (Figure]^, “religion” (e.g., church, 
mosque) and “death” (e.g., bury, kill) categories have higher 
ratios (more than 1.2, p < 0.1). This is in line with our 
findings in our previous work on profanity usage analysis in 
ask.fm social media, where we observed that there is high 
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Table 2: Mean values of social graph properties for cyberbullying versus non-cyberbullying samples and 
aggression versus non-cyberaggression. (*p < 0.05). 


Label 

*Likes 

*Media objects 

Following 

Followers 

Non-cyberbullying 

9,684.4 

1,145.7 

668.1 

415,676.2 

Cyberbullying 

7,029.0 

1,198.3 

626.7 

463,073.1 

Non-cyberaggression 

9,768.6 

1,133.7 

665.9 

421,075.3 

Cyberaggression 

7,551.3 

1,204.3 

640.3 

440,403.6 


(n 


Liguistic style 



Figure 7: Ratio of LIWC values of cyberbully- 
ing/cyberaggression labeled media sessions to non- 
cyberbullying/non-cyberaggression class in Linguis¬ 
tic categories. 

profanity usage around words like “muslim” |17| . This con¬ 
firms that religion-based cyberbullying is quite prevalent in 
social media. On the other hand, ratios for personal cate¬ 
gories like “work”, “money” and ‘achieve” are much closer 
to 1. 


Personal Concerns 



Figure 8: Ratio of LIWC values of cyberbully- 
ing/cyberaggression labeled media sessions to non- 
cyberbullying/non-cyberaggression class in Per¬ 
sonal Concerns categories. 

For psychological measurements (Figure]^, we notice that 
the ratios for “negative emotion”, “anger”, “body”, and 
“sexual” categories are significantly higher than 1 (more 
than 1.4, p < 10“®), and the ratio for “positive emotion” 
category is significantly lower than 1 (0.76, p < 10“®). 
Higher ratios for “body” (e.g. face, wear) and “sexual” (e.g. 
slut, rapist) categories provide evidence for appearance-based 
and sexual-based cyberbullying in social media. For other 
psychological measurement categories, such as “social”, “friend”, 
etc., the ratios are closer to 1. Based on our observations 


from Figures and our final important finding with re¬ 
spect to linguistic features is as follows: There is a higher 
probability of cyberbullying in postings involving religion, death, 
appearance and sexual hints, and cyberbullying posts typi¬ 
cally have higher occurrences of negative emotions and lower 
occurrences of positive emotions. 

5.4 Image Content Analysis 

We now explore the relationship between image content 
and cyberbullying/cyberaggression in a media session. Re¬ 
call that contributors could place an image in more than 
one category. More than 70% of the images were labeled 
with only one category, and around 20% were labeled with 
two categories. However, there were few images that were 
labeled with up to eight unique categories. Figureshows 
the distribution of the number of contents which have been 
assigned to the images by CrowdFlower contributors. We 
used majority mechanism to decide on a single content cat¬ 
egory for each image. In case of a tie, a graduate student 
looked at those images to break the tie. 



Figure 10: Distribution of number of unique con¬ 
tents assigned to each image in mnltichoice labeling. 

Figure shows the fraction of the contents for all la¬ 
beled data in green bar. The blue and red bars embedded 
inside each green bar in this figure indicate the fraction of 
images that belonged to the media sessions that can be cate¬ 
gorized as containing cyberaggression and cyberbullying re¬ 
spectively. The “dont know” choice was given as we realized 
that for some images it is hard to figure out what is in the 
image. As some images belong to more than one category, 
the bars will sum up to more than one. 

First, we observe that the most common labels for image 
content are Person/People, Text and Sports. Second, we 
observe that for some content categories such as “Drugs”, 
the fraction is quite small, but most of the images in those 
categories do belong to media sessions with cyberaggres- 
sion/cyberbullying in them. To understand this further, in 
Figure fT^ we show the fraction of images labeled as cyberag- 
gression/cyberbullying for each content category. We notice 
that for content category “Drugs”, 75% of the images belong 
to media sessions containing cyberbullying, while for con- 
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Figure 9: Ratio of LIWC values of cyberbullying/cyberaggression labeled media sessions to non- 
cyberbullying/non-cyberaggression class in Psychological categories. 



Figure 11: Fraction of image categories for all media sessions, cyberbullying and cyberaggression classes. 


tent categories like “Car”, “Nature”, “Person”, “Celebrity”, 
“Text” and “Sport”, 30%-40% of the images belong to media 
sessions containing cyberbullying/cyberaggression. Also, when¬ 
ever images contain bike, food, tattoo, etc., there is little cy¬ 
berbullying occurring. The key finding here is that certain 
image contents such as Drug are highly correlated with cy- 
berbulllying, while some other image contents such as bike, 
food, etc. have a very low correlation with cyberbullying. 

6. CYBERBULLYING DETECTION 

Based on the labeled ground truth data, we next proceed 
to detect occurrences of cyberbullying in Instagram media 
sessions. A majority vote criterion was employed to deter¬ 
mine whether a media session was cyberbullying/cyberagg¬ 
ression or not. To design and train the classifier, five-fold 
cross-validation was applied to the data such that 80% of the 
data was used for training in each run and 20% was used for 
testing, where we report the average. To achieve a balanced 
training data set, we over-sampled the minority class of data 
labeled as cyberbullying (29% cyberbullying class). 

Based on our analysis in Section we evaluated three 
types of features, namely those features obtained from the 
content of comments, those derived from shared media ob¬ 
jects, and those obtained from user graph properties of the 
profile owner, such as the number of followers or follow¬ 
ings. For the text features, we removed characters such as 
“!”, “>”, etc, as a preprocessing step. We hrst focused on 
unigrams and bigrams. LIWC categories are derived from 
unigrams and hence implicitly included as part of this text 


analysis. Latent Semantic Analysis (LSA) was employed 
to reduce the dimensionality of the text analysis, prevent 
over-fitting and extract the semantics as the features. LSA 
based on Singular Value Decomposition (SVD) was applied 
over the unigrams and bigrams. Next, removing stop words 
such as “and”, “or”, “for”, etc. was employed, which proved 
beneficial. Finally, each feature vector was normalized such 
that its components sum to one. 

Table illustrates the performance results for different 
features and classihers. As we see, linear SVM classifiers 
based on text-based N-grams with normalization and stop 
words removal achieve the highest recall (79%), with pre¬ 
cision 71%. Other classihers were also employed, such as 
logistic regression, decision tree and AdaBoost, but they did 
not achieve better results. 

In addition, a variety of non-text features were evaluated, 
including those features extracted from user behavior (me¬ 
dia objects, following, followers), media properties (likes, 
post time, caption) and image content. For example, by 
feeding user graph properties, media statistics and image 
content to a Naive Bayes classiher, a precision of 48% and 
recall of 60% was achieved. Adding non-text features to 
the text features for SVM or logistic regression classihers 
provided approximately the same results. Our conclusion 
is that the non-text features that we used were not able to 
meaningfully improve the performance of the cyberbullying 
detection classifier compared to the text-based features. This 
conclusion is consistent with our earlier analysis from Sec¬ 
tion |5.3| in which we observed the p-values were small for 












Image content 


Figure 12: Fraction of images which have been labeled as cyberbullying and cyberaggression for each content 
category. 


text-based features, which shows that there was significant 
differentiation between cyberbullying and non-cyberbullying 
cases whereas the non-text features in cybebullying and non- 
cyberbullying samples were not different significantly. 

We also applied the same set of features for detecting 
cyberaggression in Instagram media sessions (38% cyberag¬ 
gression). The results are very similar to the cyberbullying 
classifier. Table shows that both linear SVM and logistic 
regression provide the same F-1 measure, with recall of 80% 
and precision of 74% with a logistic regression classifier. 

7. CYBERBULLYING PREDICTION 

Our next goal is to predict whether cyberbullying will oc¬ 
cur at the time an image is posted on Instagram, without 
benefit of using any subsequent comments in the media ses¬ 
sion, as we had in the cyberbullying detection. That means 
our prediction is based only on the initial posting of the me¬ 
dia object, any image features derived from the object, and 
any properties extant at the time of posting, such as graph 
features of the profile owner. 

To perform the prediction accurately, we need to augment 
our previous set of labeled media sessions, which were pre¬ 
filtered based on text analysis to contain only those with 
profanity, to also include media sessions lacking in profan¬ 
ity. This will create a labeled data set that is independent of 
the content of text comments, namely profanity, and hence 
usable for prediction, which will not have access to comment 
contents. Therefore, we randomly selected 1164 media ses¬ 
sions from the ones with no profanity usage with the criteria 
of having more than 15 comments and labeled the new set 
by the same methodology as before. In total 80 contribu¬ 
tors worked on the quiz mode, 68 passed, 11 failed and 1 
gave up. Labeled data was obtained from 59 trusted label¬ 
ers as 9 labelers were further filtered out during the labeling 
process, because they either failed the work mode or rushed 
through their labeling process. We also labeled the image 
contents of the new set of media sessions. Table [5] shows 
the statistics related to this labeling process. Furthermore, 
suing the same methodology that we used of cyberbullying 
detection case, we only consider the media sessions whose 
labeling had a confidence level more than 60%. Overall, this 
gave us 1,142 labeled media sessions. 

Table shows the user social graph measurements for the 
labeled media sessions. We observe that p-values are all less 
than 0.1 for followers and following, which was not the case 


Table 5: Labeling process statistics. Trusted judg¬ 
ments are the ones made by trusted contributors. 


Trusted Judgments 

5,638 

Untrusted Judgments 

72 

Average Test Question Accuracy of 

82% 

Trusted Contributors 


Labeled Media Sessions per Hour 
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for the same features in Table This suggests that these 
features will be more helpful in prediction of cyberbullying 
than in detection. 

Table 6: Mean values of social graph properties for 
cyberbullying versus non-cyberbullying samples and 
cyberaggression versus non cyberaggression, {**p < 
0.05, *p < 0.1). 


Label 

*Media 

objects 

^Following Followers 

Non-cyberbullying 

1,157.8 

721.7 

**398,283.' 

Cyberbullying 

1,198.3 

626.5 

**465,376. 

Non-cyberaggression 

1,152.6 

724.4 

*393,901.6 

Cyberaggression 

1,204.3 

640.3 

*440,403.6 


The examined features include image contents, media ses¬ 
sion properties, such as post time and caption, and profile 
owner social graph properties, such as the number of fol¬ 
lowers and following and total number of shared media ob¬ 
jects. A similar methodology as the detection was applied 
for training and testing of the predictor for a total 3,096 
media sessions (18% cyberbullying class). Table shows 
that cyberbullying incidents can be predicted with recall 76% 
and precision 62%, using only the image contents, media 
and user meta data based on a Maximum-Entropy (MaxEnt) 
classifier. Eor this augmented complete data set, we observe 
that these non-text features provide the main utility for pre¬ 
diction, whereas in detection, the main utility is derived from 
text-based features over the profanity-based filtered data set. 

In addition, we were interested to explore if prediction 
could be improved using only a limited set of early com¬ 
ments, not the complete set of comments for a media ses¬ 
sion. Hence we also give in Table the results of predic- 
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Table 3: Cyberbullying detection’s classifier performance 


Features 

Classifier 

Fl-measure 

Precision 

Recall 

Unigram-stopword removal 

linear SVM 

0.74 

0.70 

0.79 

Unigram -Bigram- stop word removal 

linear SVM 

0.75 

0.77 

0.73 

Unigram -Bigram-stop word removal-Normalization 

linear SVM 

0.75 

0.71 

0.79 

Unigram -Bigram-stop word removal-Normalization 

logistic regression 

0.74 

0.72 

0.75 


Table 4: Cyberaggression detection’s classifier performance 


Features 

Classifier 

Fl-measure 

Precision 

Recall 

Unigram -Bigram-stop word removal-Normalization 

linear SVM 

0.77 

0.75 

0.79 

Unigram -Bigram-stop word removal-Normalization 

logistic regression 

0.77 

0.74 

0.80 


tion using the first 5, 10 and 15 comments. We see that 
as more comments arrive, our predictor is able to improve 
its precision and recall to 72% and 77% respectively using 
a MaxEnt classifier. This suggests that text-based features 
can improve classifier performance. We hope to leverage this 
observation in future work to improve predictor performance 
by utilizing the history of comments from other previously 
shared media sessions in the prediction. 

Table [^provides the results for prediction of cyberaggres¬ 
sion media sessions. The Fl-measure is 6-9% lower than for 
cyberbullying prediction using a MaxEnt classifier, due to 
significantly lower recall. 

8. FUTURE WORK 

While this paper has advanced the understanding of cy¬ 
berbullying in a media-based mobile social network, there 
remain a number of areas for improvement. One theme for 
future work is to improve the performance of our classifier. 
New algorithms should be considered, such as deep learning 
and neural networks. More input features should be evalu¬ 
ated, such as new image features, mobile sensor data, etc. A 
limitation of our current classifier is that it is designed only 
for media sessions that have at least one profanity word. 
A more general classifier that can apply to all media ses¬ 
sions is needed. Incorporating image features needs to be 
automated by applying image recognition algorithms. Tem¬ 
poral behavior of comments for a posted media object shows 
different behavior for cyberbullying class, which should be 
taken into account in designing the detection classifier. 

In this work we have only considered the image content 
and image and user metadata for prediction of cyberbully¬ 
ing. However, based on the improvement seen in using a 
small number of text comments, we think that considering 
the commenting history of users in previously shared media 
can prove to be useful. 

Another theme for future work is to obtain greater detail 
from the labeling surveys. Our experience was that stream¬ 
lining the survey improved the response rate, quality and 
speed. However, we desire more detailed labeling, such as 
for different roles in cyberbullying - identifying and differ¬ 
entiating the role of a victim’s defender, who may also spew 
negativity, from a victim’s bully or bullies. 

9. CONCUUSIONS 

This paper makes the following major contributions: (1) 
an appropriate definition of cyberbullying that incorporates 


both frequency of negativity and imbalance of power ap¬ 
plied in large-scale labeling, and is differentiated from cy¬ 
beraggression; (2) cyberbullying is studied in the context 
of a media-based social network, incorporating both images 
and comments in the labeling; (3) a detailed analysis of the 
distribution results of labeling of cyberbullying incidents is 
presented, including a correlation analysis of cyberbullying 
with other factors derived from images, text comments, and 
social network meta data; (4) multi-modal classification re¬ 
sults are presented for cyberbullying detection as well as 
prediction that incorporate a variety of features to identify 
cyberbullying incidents. 

The major findings of this paper comprise the following 
results. First, we found that labelers are mostly in agree¬ 
ment about what constitutes cyberbullying and cyberag¬ 
gression in Instagram media sessions. Second, we found a 
significant number of media sessions containing profanity 
and cyberaggression that were not labeled as cyberbullying, 
suggesting that classifiers for cyberbullying must be more 
sophisticated than mere profanity detection. Third, media 
sessions with very high percentage of negativity above 60- 
70% actually correspond to a lower likelihood of cyberbully¬ 
ing. Fourth, media sessions with cyberbullying exhibit more 
frequent commenting. Fifth, users of media sessions contain¬ 
ing cyberbullying demonstrate a lower number of likes per 
post. Sixth, cyberbullying has a higher probability of oc¬ 
curring when media sessions contain certain linguistic cat¬ 
egories such as death, appearance, religion and sexuality 
content. Seventh, certain image contents such as “drug” are 
highly related to cyberbullying while other image categories 
such as “tattoo” or “food” are not. Eighth, a linear SVM 
classifier can detect cyberbullying with 79% recall and 71% 
precision using only text features, where non-text features 
were not helpful. Ninth, however, non-text features such as 
image and user meta data were central to cyberbullying pre¬ 
diction, where a MaxEnt classifier achieved 76% recall and 
62% precision. 
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